Running head: JUST-IN-TIME, SCHEMATIC SUPPORTIVE INFORMATION PRESENTATION

Cognitive load theory states that well-designed learning material minimizes extraneous cognitive load and optimizes germane cognitive load within the thresholds of available cognitive resources. In this study, the extraneous cognitive load is minimized by avoiding temporal split attention with regard to supportive information (i.e., conceptual models or 'theory') and the germane cognitive load is optimized by using schematic representations of this information to direct learner’s attention to concepts relevant for learning. A 2x2 between-groups design with the factors supportive information (before or during practice) and schematic representation (before or during practice) was used to investigate whether this balance between extraneous and germane load leads to more effective and efficient learning. It was found that the 'supportive during, schema before' format indeed yielded a higher learning efficiency than the 'supportive before, schema before' and the supportive during, schema during' format but no differences were found for learning effectiveness (i.e., test performance). 2 Just-in-time, schematic supportive information presentation Just-in-time, Schematic Supportive Information Presentation during Cognitive Skill Acquisition Practical educational approaches, such as project-based education, the case method, problem-based learning, and competency-based learning usually focus on (realistic and authentic) whole tasks or meaningful problems (Merrill, 2002; Reigeluth, 1999; van Merriënboer & Kirschner, 2001). Such tasks and problems help learners master the necessary cognitive skills and transfer what is learned in school to their daily life or work settings. A potential pitfall of using such tasks is that, because of their complexity, they demand too much of the learner’s cognitive system. Working memory capacity is limited and exceeding this capacity hampers learning (Baddeley, 1992; Miller, 1956; Sweller, 1988). The load imposed on working memory can be influenced in several ways. First, working on realistic tasks can be supported by sequencing tasks during practice from simple to complex (Reigeluth, 1983; 1999) or by using low-load task formats, such as worked-out examples (Paas & van Merriënboer, 1994; Van Gerven, Paas, van Merriënboer, & Schmidt, 2002) or completion tasks (van Merriënboer, Schuurman, de Croock, & Paas, 2002; van Merriënboer & de Croock, 1992). Second, information necessary to carry out the tasks can be presented just-in-time; in other words, precisely when the learner needs it for practice (van Merriënboer, Kirschner, & Kester, 2003; Kester, Kirschner, & van Merriënboer, in press). These instructional interventions aim at optimally utilize working memory capacity to facilitate cognitive skill acquisition. The latter approach is the focus of the current study. Cognitive Load The cognitive load that a learner experiences during learning and practice is caused by a combination of the complexity of the problem to be solved and the design of the instructional materials. Solving the problem yields intrinsic cognitive load which is determined by the number of elements in the instructional material (e.g., practice problems, subject-relevant information) and the degree of element interactivity. Low element interactivity is associated with low intrinsic cognitive load because it allows the learner to serially process a small number of elements at a 3 Just-in-time, schematic supportive information presentation time (Sweller, van Merriënboer, & Paas, 1998). For example, it is easy for a learner to serially process the meaning of the symbols used in a formula that is used to calculate the weighed input in a simple neural network. High element interactivity, on the other hand, is associated with a high intrinsic cognitive load because it requires the learner to process several elements and their relationships in working memory simultaneously in order to solve the problem. In this study, the learner has to simultaneously process several features of simple neural networks such as input nodes and output nodes to train such a system in categorizing input patterns. The design of the instructional material yields both extraneous cognitive load and germane cognitive load. Extraneous cognitive load is caused by those processes a learner engages in that are not directly beneficial to learning, for instance, searching for relevant information sources, combining different information sources in order to understand the learning material, or weak-method problem solving. Extraneous cognitive load uses up cognitive resources at the cost of learning processes. Germane cognitive load includes all cognitive load associated with processes a learner engages in that are beneficial for learning, for instance, attending to important features of the problem (van Merriënboer, Schuurman, de Croock, & Paas, 2002), or abstracting from a variety of practice problems so as to construct more general cognitive schemata (Spiro, Coulson, Feltovich, & Anderson, 1988; Sweller et al., 1998). With a given intrinsic cognitive load, well-designed learning material minimizes extraneous cognitive load and optimizes germane cognitive load within the thresholds of totally available cognitive resources. Within the broad range of different strategies available for managing the various forms of cognitive load (Chandler & Sweller, 1991; Sweller, 1988; Sweller et al., 1998), the present study focuses on the following strategies: (1) managing intrinsic cognitive load by sequentially presenting different training elements (e.g., information and practice tasks), (2) minimizing extraneous cognitive load by just-in-time and just-in-place information presentation to avoid split attention, and (3) optimizing germane cognitive load by directing 4 Just-in-time, schematic supportive information presentation learners' attention to concepts relevant for learning by providing schematic representations of information necessary to carry out the tasks. Managing intrinsic cognitive load In this study, a simulation is used to teach learners the functioning of a simple, two-layer neural network. Training such a simulated neural network to categorize input is the cognitive skill that learners have to master. Such cognitive skills are made up of different constituent skills (Fisk & Gallini, 1989; van Merriënboer, 1997; van Merriënboer et al., 2003). Non-recurrent constituent skills require interpretation of cognitive schemata, and the performance of such skills varies from problem situation to problem situation. In other words, it is the situation-specific use of the same, general knowledge. In the present study, an example of a non-recurrent skill is being able to reason about the working of a simple neural network in the case of an ANDclassification-problem. Recurrent constituent skills require applying cognitive rules or automated schemata and the performance of these skills is the same in every problem situation. In other words, it entails the situation-independent use of the same, situation-specific knowledge. An example of a recurrent skill in this study is calculating the total input of a simple neural network by means of the appropriate formula. Non-recurrent and recurrent skills constitute the total cognitive skill that allows learners to solve problems in the domain. In order to master the non-recurrent constituent skills, supportive information such as a conceptual model of how a learning domain is organized, is necessary. Learning supportive information is, in general, a task with high element interactivity because to-be-constructed mental models contain many interrelated elements. This has consequences for the proper management of cognitive load during practice (van Merriënboer et al., 2003). The foregoing discussion of cognitive load yields the advice to present supportive information before the practice problems since simultaneously solving the practice problems and processing the highly interactive supportive information is likely to increase the risk of overloading working memory. When supportive information is presented before practice, all cognitive capacity is available to 5 Just-in-time, schematic supportive information presentation process this information, which allows learners to construct rich and adequate cognitive schemata. As a result, cognitive schemata that can easily be activated in working memory during the solution of practice problems will be available in long-term memory. On the other hand, mastery of recurrent constituent skills requires the availability of procedural information; task-specific rules along with the facts, principles, or concepts needed to correctly apply these rules. Learning procedural information is, in general, a task with low element interactivity because task-specific rules always contain a limited number of related elements (i.e., some conditions and one action). Presenting procedural information during practice is thought to be warranted, because the risk of overloading working memory when simultaneously processing procedural information and solving practice problems is low due to the low element interactivity of the procedural information. In sum, to properly manage intrinsic cognitive load it is important to sequentially present the supportive information and the practice tasks while simultaneously presenting the procedural information and the practice tasks. Minimizing extraneous cognitive load Extensive research has been carried out regarding the split attention effect (for an overview see Sweller et al., 1998; Kester, Kirschner, & van Merriënboer, in press). This research provides guidelines on when to (i.e., avoidance of temporal split attention), and how to (i.e., avoidance of spatial split attention) present the procedural and supportive information. Mayer and his colleagues (Mayer & Anderson, 1991; Mayer & Moreno, 1998; Mayer & Sims, 1994) showed that temporal integration of narration and animation results in lower extraneous cognitive load because learners do not have to ‘search-and-match’ and mentally integrate the (narrated or animated) information that is presented. Translated to the present study, this could mean that although the supportive information and the practice task have to be sequentially presented in order to avoid cognitive overload, the time lapse between relevant supportive information and the practice task should be minimal to avoid temporal split-attention. 6 Just-in-time, schematic supportive information presentation With regard to the procedural information, Sweller and his colleagues (Chandler & Sweller, 1991; Sweller, 1994) showed that extraneous cognitive load is reduced significantly by integrating explanatory text (i.e., here procedural information) in a diagram, instead of separating the diagram from the explanatory text spatially. Spatially integrating text and diagram frees learners from the need to mentally integrate the different information sources themselves which, in turn, lowers extraneous load. In this study the procedural information, that is the information necessary to carry out a specific task, accompanies graphical representations of neural networks. To avoid spatial split-attention and, thus, to minimize the extraneous cognitive load in the instructional material, it appears to be advisable to present procedural information during practice, integrated in the graphical representations of the simple neural networks (Kester, Kirschner, van Merriënboer, & Bäumer, 2001; van Merriënboer et al., 2003). Moreover, the simultaneous, integrated presentation of procedural information and practice tasks helps learners to automate schemata that they apply to familiar aspects of novel problem situations, because the necessary procedural information is made directly available in working memory during practice and so facilitates a process known as knowledge compilation (Anderson, 1996), by which the relevant knowledge is compiled in automated schemata. Thus, in order to minimize extraneous cognitive load by avoiding temporal and spatial split-attention, it is important to make the time lapse between the supportive information and the practice task not too long and present the procedural information in an integrated format during the practice tasks (i.e., just-in-time and just-in-place). Optimizing germane cognitive load It is not, however enough to only minimize extraneous cognitive load. Instructional materials must also optimize germane cognitive load. To support learners in constructing rich and adequate schemata during task practice, schematic representations of the supportive information can be used. The idea that schematic representations direct a learner's attention to concepts relevant for learning and thus, facilitate learning originates from Ausubel’s 7 Just-in-time, schematic supportive information presentation Subsumption Theory (1963). According to Ausubel, meaningful learning only occurs when new material can be appropriately subsumed under a relevant, existing concept. In order to facilitate this process, advance organizers (e.g., in the form of schematic representations of learning material) can be applied in instructional design. These organizers provide advance ideational scaffolding: when the learner is confronted with unfamiliar material, a cognitive structure is available for incorporating the new concepts (Ausubel, 1963; Jonassen, 1982). In a meta-analysis of 132 studies, advance organizers appeared to facilitate learning and retrieval (Jonassen, 1982; Mayer, 1979). However, they have no effect when relevant cognitive structures are already available and are appropriately activated by either the learner or the material presented. Nowadays, the advance organizers are considered to be a type of graphic organizer (i.e., figural organizations of text information, Robinson & Kiewra, 1995; Robinson, 1998) used to promote meaningful learning. Graphic organizers can take many forms (e.g., matrices or tree diagrams) but an important aspect they all have in common is that they provide hierarchical and coordinate relations (Robinson & Kiewra, 1995; Robinson, Katayama, Dubois, & Devaney, 1998). Thus, to optimize germane cognitive load, schematic representations (or graphic organizers) of the supportive information can be used to direct learners' attention to concepts relevant for schema construction, provided that the supportive information is subsequently available in the instructional material. The focus of this study is on the presentation of supportive information with aim of shedding some light on minimizing extraneous cognitive load by avoiding temporal splitattention with regard to the supportive information. Moreover, a closer look is taken at the role of using schematic representations of the supportive information to optimize germane cognitive load. To investigate these two aspects, the timing of the presentation of the supportive information (before or during practice) and the schematic representation (before or during practice) is varied. The procedural information is presented during practice in all conditions. Practice consists of the presentation and solution of a series of practice tasks dealing with neural 8 Just-in-time, schematic supportive information presentation networks. Presentation before practice means that the supportive information and/or the schematic representation are available only before the practice task. Presentation during practice means that the supportive information is available during each practice task and/or the schematic representation is available during each practice task. With respect to learning effectiveness, it is hypothesized that making supportive information available during practice and the schematic representation before practice will yield higher test performance than the other conditions, since minimizing extraneous cognitive load and optimizing germane cognitive load during learning facilitates the acquisition of rich, adequate schemata. Furthermore, learning will be more efficient when supportive information is presented during practice and the schematic representation is given in advance. In other words, a relatively high test performance will be obtained with relatively low cognitive load during practice. Optimally balancing extraneous and germane cognitive load enables learners to use a maximum of their cognitive capacity for processes relevant for learning because little cognitive load is caused by activities irrelevant to learning. Method Participants Forty-eight secondand third-year students (26 male and 22 female; mean age = 23.5 years, SD = 3.1) of the Faculty of Psychology at the Maastricht University voluntarily participated in this study. During a practice session they had to learn to train a simple neural network, using a computer-based simulation. All participants followed a comparable practical at least one year earlier. Before the experiment the participants received a 12-item, multiple-choice, factual knowledge test to assess their prior-knowledge and they had to fill in the grade they received the first time they carried out the practical. An ANOVA showed no significant differences between the experimental groups on the pre-experimental factual knowledge test, F(3, 48) = .31; p = .82. Moreover, the participants achieved low scores on this test (M = 2.62; SD 9 Just-in-time, schematic supportive information presentation = 1.86; max. = 12) which indicated that they al had low prior knowledge about the problem domain. A Kruskal-Wallis test on the ordinal grade data showed no significant differences between the experimental groups, χ(3, 48) = 1,48; p = .69. Materials Adaline. A computer-based simulation program Adaline was used to teach students the functioning of a simple, two-layer neural network. Adaline stands for 'ADAptive LINear Element'. The network contains nodes, which are simplified representations of neurons, and weights, which are simplified representations of synaptic connections in the brain. The function of this simulated neural network is to categorize a set of nodes that form a specific input pattern. This input pattern is represented by an input vector in combination with a weight vector (i.e., the strength of synaptic connections) in a coordinate system. Adaline has to be ‘trained to recognize’ input vectors as belonging to one of two categories, A or B. This is done by adjusting the weights of the network through vector processing. After proper training, the network is able to solve AND/OR classification problems. Adaline is part of the Psychology curriculum at Maastricht University and is used as a practical for second-year students in an introductory course on cognitive science. Here, Adaline is adapted to the experimental purposes and it starts with an introduction in which vector processing is explained. The introduction is followed by 11 practice tasks. Information presentation. The practice tasks, consisting of simulated simple neural networks, were accompanied by supportive information, a schematic representation of this information, and procedural information. The procedural information was presented stepwise during practice for all experimental groups. The supportive information and the schematic representation were made available either before practice, during practice or before and during practice. Four information-presentation formats were distinguished in a 2 x 2 factorial design with the factors and timing of supportive information, before or during practice, and timing of schematic representation presentation presentation, before or during practice. In the 'supportive 10 Just-in-time, schematic supportive information presentation before, schema before' format both the supportive information and the schematic representation were presented before practice. The participants assigned to the 'supportive during, schema during' format received both the supportive information and the schematic representation during practice. In the 'supportive before, schema during' format the supportive information was presented before practice while the schematic representation was presented during practice. Finally, in the 'supportive during, schema before' format the supportive information was presented during practice while the schematic representation was presented before practice. For an example of a practice problem see Figure 1. *** Insert Figure 1 about here*** In the first two practice tasks in every format, the participants had to read through the supportive information and study the schematic representation before they could fill in the answers to the practice tasks. In the remaining nine practice tasks the supportive information and the schematic representation were available on demand by clicking a 'more information' button or a 'schema' button. Practice tasks. Adaline contains 11 practice tasks that the participants have to carry out in a fixed order. In these practice tasks, participants have to perform a series of calculations (i.e., vector processing) to categorize the input pattern. First, they have to calculate the weighed input which results in a weighed input vector. Second, they have to define the output. That is, if the weighed input > 0, then the output is 1 (i.e., category A); if the weighed input is ≤ 1, then the output is –1 (i.e., category B). Third, they have to calculate the error, that is, they have to check whether the expected output (i.e., this is given) is the same as the real output defined in step 2. If this is not the case, then they have to perform a fourth step in which they have to adjust the weighed vector by applying the delta rule (i.e., a mathematical procedure that calculates the minimal error) until it fits the input as much as possible. Eight tasks only required the first three steps and three tasks also required the fourth step. 11 Just-in-time, schematic supportive information presentation Factual knowledge test. Before and after the experimental session in Adaline (i.e., the 11 practice tasks) the same multiple-choice test was administered to the participants. Before the experimental session this test was used to measure participants’ prior factual knowledge while after the experiment it was used to measure factual learning or retention. This test contained 12 items and the participants could choose their answer from four alternatives. Comprehension test. After the experimental session and the factual knowledge test a comprehension test was administered to the participants. This test contained four open questions and was meant to test whether the participants had gained insight in the working of simple, twolayer neural networks. They could receive one point for each correctly answered open question, so the maximum score on the comprehension test was four. To determine the inter-rater reliability, the comprehension test scores of ten participants were determined by two raters. The inter-rater reliability for the comprehension test scores of the two raters was .93 (Intraclass Correlation Coefficient). Workload measurement. The NASA-TLX tool was used to measure workload (1) after the practice tasks, and (2) after the factual knowledge test (post experiment) and the comprehension test. The NASA-TLX is a subjective workload assessment tool that involves a multi-dimensional rating procedure, from which an overall workload score based on a weighted average of ratings on six subscales (i.e., mental demand, physical demand, temporal demand, performance, effort and frustration) is derived. Except for the performance scale, which ranged from 'good' to 'poor', all scales ranged from 'low' to 'high' and were divided in twenty sections (see Figure 2a). The participants had to click on one of the twenty sections that best represented their experience for each aspect. To establish the weighing, 15 possible pair-wise comparisons of the six scales were presented to the participants (see Figure 2b). The participants had to select the member of each pair that contributed more to the workload of that task than the other member (e.g., temporal demand vs. frustration or performance vs. mental effort). Based on the scale-scores and the 12 Just-in-time, schematic supportive information presentation weighing, the total workload was calculated (minimum workload = 5; maximum workload = 100). ***Insert Figure 2a, b about here*** Learning and performance efficiency. The procedure of Paas and van Merriënboer (1993; see also Paas et al., 2003) was used to calculate efficiency. First, the performance measures (i.e., the factual knowledge test and the comprehension test) and the workload measures for each participant are transformed into z-scores, using the grand mean across conditions. Then, the mean z-scores for every condition are represented in a Cartesian coordinate system with Workload (W) on the horizontal axis and Performance (P) on the vertical axis. The line P = W through the origin of the system is assumed to indicate a neutral efficiency (slope = 45°). The efficiency, E, is calculated as the perpendicular distance from a data point in the coordinate system to the line P = W (Paas & van Merriënboer, 1993). The formula for calculating this distance is: Performance Workload E =